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ABSTRACT 

This issue brief reports on accountability policies and 
practices for elementary students with emotional and behavioral disorders in 
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2001 and March 2002, a study was conducted of these schools, gathering 
responses from 271 principals and 229 teachers. Major findings included: (1) 

the majority of respondents used assessments required by the local district 
and/or state as their primary accountability tool. However, approximately 30% 
of teachers and 20% of principals relied on teacher-selected assessments and 
another 11% used assessments developed by the school; (2) more than half of 
the teachers and principals whose schools used local district and/or state 
assessments reported that between 81-100% of students participated; (3) 
respondents from 31 states reported that state standardized norm-referenced 
or criterion-referenced assessments were available as alternate assessments; 
(4) more than 80% of respondents reported a school assessment accommodation 
policy, most often based on state or district guidelines; (5) assessment 
results were most commonly reported to parents, guardians, and teachers, and 
maintained in each student * s file; (6) assessment results were most often 
used to adjust instruction or curriculum and identify areas where school 
performance was acceptable or needed improvement. (Contains 18 references.) 
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Educational Accountability in Day Treatment and 
Residential Schools for Students with 
Emotional and Behavioral Disorders: 

Report on a National Survey 

Issue Brief Highlights 

This Issue Brief reports on accountability policies and practices for elementary students 
with emotional and behavioral disorders (EBD) in private and public day treatment and resi- 
dential schools. Between January 2001 and March 2002, researchers conducted a nationally 
representative survey of these schools, gathering responses from 271 principals and 229 
teachers. 

Based on previous research (Hollenbeck, Tindal, & Almond, 1998), the authors expected 
significant differences in the ways teachers and principals answered survey questions. 
However, for the 216 schools that had both principals and teachers return surveys, there were 
no statistically significant differences in responses. 

The survey s major findings were related to a number of school-level assessment issues: 

• Determining Accountability - The majority of respondents used assessments required by 
the local district and/or state as their primary accountability tool. However, approximately 
30% of teachers and 20% of principals relied on teacher-selected assessments, and anoth- 
er 11% of participants used assessments developed by the school. 

• Participation in Assessments - Over half of the teachers and principals whose schools 
used local district and/or state assessments reported that between 81-100% of their stu- 
dents participated. Yet a large number of schools reported 80% or fewer students partici- 
pated, highlighting a need for schools to collaborate with local and state education agencies. 
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• Alternate Assessments - Respondents from 31 states reported that state standardized norm- 
referenced or criterion-referenced assessments were available as alternate assessments. 
Schools also used teacher-made assessments and a small number of participants reported 
no alternate assessments available at their schools. 

• Accommodations - Over 80% of teachers and principals reported a school assessment 
accommodation policy, most often based on state or district guidelines. Approximately 
20% of respondents said they offered no assessment accommodations. 

• Reporting Assessment Results - Assessment results were most commonly reported to par- 
ents, guardians, and teachers, and maintained in each student’s file. Relatively few princi- 
pals and teachers reported results to districts or states. 

• Using Assessment Results - According to respondents, assessment results were most often 
used to adjust instruction or curriculum and identify areas where school performance was 
acceptable or needed improvement. A relatively large number of schools used the data to 
make decisions regarding students’ return to public or home schools. 

As the survey results indicate, more effort is needed to assure that students with EBD in day 
treatment and residential settings participate in assessments with appropriate accommoda- 
tions, that alternate assessments are utilized only when necessary, and that assessment data are 
properly reported and used. 
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Access to challenging curriculum and accountability for student learning provides the 
framework for current educational reform. One important component of assuring access to 
the general education curriculum is evaluating how well students are acquiring the skills and 
content taught (Elliott. Erickson. Thurlow. & Shriner, 2000; Elliott, Kratochwill, & Schulte, 
1998; Hehir, 1999). Recent federal educational reform initiatives, such as the No Child Left 
Behind Act, advance the issue of school accountability. A primary component of this mandate 
is increased accountability for student learning through regular high-stakes assessments and 
reporting of results (U.S. Department of Education, 2002). 

For students with disabilities, the Individuals with Disabilities Education Act (IDEA) (1997) 
is the primary driving force behind assuring access to the general education curriculum and 
educational accountability. Despite a national trend toward inclusion of students with disabili- 
ties in general education classes and accountability systems, students with emotional and 
behavioral disorders (EBD) often have difficulty remaining part of the mainstream education- 
al environment (Kauffman, 1997; Muscott, 1997) and are placed in exclusionary settings that 
may offer greater behavioral and therapeutic support than the regular public school. For stu- 
dents ages 6-21 who are served under IDEA, those with EBD are more likely to be placed in 
restrictive settings than youth in any other disability classification (U.S. Department of 
Education, 2001). Currently, more than 77,000 students with EBD are educated in separate 
day treatment or residential schools (U.S. Department of Education, 2001). In light of the 
requirements of The No Child Left Behind Act and the current IDEA mandates for increased 
accountability and access to the general education curriculum, it is critical to identify the 
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extent to which school-level accountability policies in day treatment and residential schools 
are linked to district and state policies. Researchers conducted a survey to investigate issues 
related to school-level policies, as well as the characteristics of students, teachers, principals, 
and programs (see Gagnon, 2002) (see Figure 1). 



Figure 1 - Research Questions for National Survey 

To develop a systematic description of current school-level policies in 

elementary-level day treatment and residential schools for students with 

EBD, investigators addressed the following questions: 

1 . What are the characteristics of teachers, students, and principals and 
how do these compare across program type, organizational structure, 
and accreditation? 

2. What are the characteristics and policies of the educational programs? 

a. What are the curricular policies? 

b. What are the policies related to educational accountability? 

c. What are the policies designed to facilitate student entrance 
into and exit from the program? 

d. What are the general program characteristics (e.g., philosophy, 
who sets school policy, accreditation, types of services, organization- 
al structure, grade levels, number of students in a class)? 

3. Do policies and percentage of students participating in district or state 
assessment relate to student and program characteristics? 

4. What is the relative emphasis on instruction versus behavioral/therapeutic 
issues within the educational program? How does this emphasis vary across 
types of services offered (day treatment, residential, combined day treatment 
and residential), organizational structure (public school, private non-profit 
school, private for-profit school), and accreditation (yes or no)? 

5. Do reports of school policies relate to respondent's role as teacher 
or principal? 



This Issue Brief focuses on the accountability issues addressed in the survey, including: 

• Primary accountability and participation in assessments; 

• Assessment accommodations; 

• Alternate assessments; 

• Reporting of assessment results; 

• Use of assessment results. 
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Method 



Due to the limited research on school-level policies in day treatment and residential 
schools, investigators used the survey to explore and identify current national trends. The 
study included a random sample of private and public day treatment and residential schools 
for students with EBD that serve any of grades 1-G. The study was restricted to these grades 
because the goal for most elementaiy-age students in day treatment or residential schools is 
to transition back to their home or public school (Grosenick, George, & George, 1987). 
Although information is limited, most students in such placements do transition to a less 
restrictive setting within traditional schools (Baenen, Glenwick, Stephens, Neuhaus, & Mowrey, 
1986; Gagnon & Leone, 2002). For this transition to be successful students must have continu- 
ous access to curriculum and instruction that is based on general education standards. 

Because no comprehensive national list of schools met the criteria for inclusion in the 
study, researchers completed the following steps to identify the sample: 

• Acquired a commercial database of special education schools, alternative schools, and 
alternative programs from Market Data Retrieval (n = 6,110); 

• Chose a random sample of schools (n = 4,000); 

• Verified via phone interview that each selected school met the criteria (day treatment or 
residential school for students with EBD including any of grades 1-6). 

This process yielded a total of 636 schools. However, 156 survey respondents noted that 
their school had been inaccurately classified as a day treatment or residential school program 
for children with EBD. Therefore, the final sample was 480 schools. 

To increase the validity of the survey instrument and allow for greater generalizability, the 
investigators developed both a teacher survey and principal survey based on a review of liter- 
ature, consideration of current educational reform, discussion with experts in the field of 
special education, and separate teacher and principal focus groups. Possible threats to relia- 
bility were addressed through the standardization of the survey format, directions, and 
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questions (Fink, 1995). Also, reliability checks were conducted on data entered for 30% of 
teacher surveys and 30% of principal surveys. Agreement was calculated by dividing the 
number of agreements by the number of agreements and disagreements x 100. Reliability 
was 99.87% for teacher surveys and 99.86% for principal surveys. 

Data Collection 

Data was collected between January 2001 and March 2002. The principal and one ran- 
domly selected teacher were surveyed from each randomly selected school. Initially, the 
principal was mailed an introductory letter. The first survey mailing included a cover letter, 
principal survey, and teacher survey, with a $2.00 bill attached to each survey. The cover let- 
ter requested that principals make an alphabetical list of teachers who instruct students in 
the first through sixth grades. The principal was directed to give the survey to the first 
teacher on the alphabetized list. Three weeks after the first mailing, a second occurred. 
Principals received surveys and a letter requesting a response from either the teacher or 
principal, or both. At the same time, nonrespondents were urged by phone to complete the 
survey. A third and final mailing occurred three weeks after the second mailing. 

Results 

Results are organized under five general categories: (a) respondents versus nonrespon- 
dents; (b) school policies based on respondent roles; (c) primary accountability and partici- 
pation in assessments; (d) alternate assessments and assessment accommodations; 

(e) reporting and use of assessment results. 



Respondents Versus Nonrespondents 



A total of 271 principals (56.45%) and 229 teachers (47.7%) returned their surveys. Of 
those, 216 schools (44.58% of the total) had both teacher and principal return surveys. The 
sample of 480 included schools from 48 states and the District of Columbia. All census 
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bureau regions were represented and the return-rate percentages from each region were generally 
similar to the percentage of national population from each region (U.S. Census Bureau, 2001). 

Respondent and nonrespondent comparisons were completed using school-level data from 
the Market Data Retrieval database. Nonrespondents in the sample were compared to those 
who responded on locale (i.e., urban, suburban, rural), census bureau region (e.g.. 
Northeast, Midwest, South, West), school enrollment (e.g., 1-99, 100-199, 200 or more), 
school type (e.g., alternative education schools, alternative education programs, or special 
education schools), and organizational structure (e.g., public schools or the combined cate- 
gory of county, state, private, non-Catholic, and Catholic schools). Teachers and principals 
from each school were surveyed separately. Thus, three types of respondent and non-respon- 
dent comparisons were conducted: (a) schools with only teacher respondents versus schools 
with only principal respondents and schools with neither teacher nor principal respondents; 
(b) schools with only principal respondents versus schools with only teacher respondents 
and schools with neither teacher nor principal respondents; and (c) schools with both 

teacher and principal respondents versus schools with neither teacher nor principal respon- 

2 

dents. Only school type was significant ( X = 26.179, 2, p < .01) when comparing propor- 
tions of schools in which both teacher and principal surveys were returned and schools in 
which no surveys were returned. One hundred and ninety-seven special education schools 
had surveys returned by the teacher and principal. Fewer alternative education schools (n = 
12) and alternative education programs (n = 7) had both surveys returned. With this excep- 
tion, the sample appeared to be nationally representative of day treatment and residential ele- 
mentary schools for students with EBD. 
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School Policies Based on Respondent Roles 



Based on previous research (Hollenbeck, Tindal, & Almond, 1998), investigators anticipat- 
ed variations in perceptions of school policy between teachers and principals. However, for 
the 216 schools in which both principals and teachers returned surveys, no statistically signif- 
icant dilferences in responses were noted. Given the large number of policy questions, this 
was a critical finding. A national picture emerged showing that teachers and principals were 
clearly "on the same page" with regard to school-level policy. Also, the consistency of 
responses across teachers and principals, coupled with the random selection of participants, 
demonstrated that the school-level policies reported throughout this study are an accurate 
representation of day treatment and residential school policies in the U.S. 



Primary Accountability and Participation in Assessments 

Teachers and principals were asked to identify their schools primary means of accountabil- 
ity for student performance (see Table 1). 



Table 1 - Primary Accountability and 
Participation in Assessments 

Policy and Participation Teacher Response* Principal Response* 

No. (%) No. (%) 

Primary Accountability forAcademic Performance: 

Individual teachers select how to 



assess student performance 


66 


(29.7%) 


56 


(22.2%) 


Assessments required by local 
district and state 


131 


(59.0%) 


167 


(66.3%) 


School developed assessment 


25 


(1 1 .3%) 


29 


(1 1 .5%) 



Students Participating in 
District and State Assessment: 



20% or less 


22 


(15.9%) 


17 


( 9.2%) 


21-40% 


12 


( 8.7%) 


13 


( 7.1%) 


41-60% 


19 


(13.8%) 


14 


( 7.6%) 


61-80% 


4 


( 2.9%) 


21 


(1 1 .4%) 


81-100% 


81 


(58.7%) 


119 


(64.7%) 



Note: • Teacher responses based on class data, principal responses on school data. 
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Teachers and principals most commonly reported using assessments required by the local 
district and/or state as the basis for school accountability. Less frequently reported was a poli- 
cy that permitted individual teachers to select how to assess student performance. 

Teachers and principals who stated that school accountability policies were based primarily 
on local district and/or state assessments were also asked to identify the percentage of stu- 
dents who participated (see Table 1). Most frequendy, teachers and principals reported 81- 
100% of students participated in district and state assessments. However, 15.9% of teachers 
and 9.2% of principals reported that 20% or fewer of students participated. 

Based on the principal responses, student participation in district and state assessments dif- 
fered significantly among schools serving different student populations (e.g., students from 

2 

within a district, across a single state, or from more than one state) ( X = 13.668, 3, p < .01). 
Schools in which 61% or more of students participated in district and state assessments com- 
monly served students from a single district (n = 47, 26.7%) or from within their state (n = 
52, 29.5%) (Note that in this analysis, percentages reported indicate chi-square cell size). 

Alternate Assessments and Assessment Accommodations 

Teachers and principals who indicated that district and state assessments were their pri- 
mary accountability tool (n = 131, 59.0%; n = 167, 66.3%) were also questioned on school 
policies concerning alternate assessments and assessment accommodations. Of these, 57 
teachers and 76 principals responded that state standardized norm-referenced or criterion 
referenced assessments were used or available as alternate assessments. The next most com- 
mon response was teacher-made assessments (n = 47 and n = 57, respectively). In addition 18 
teachers and 17 principals reported that no alternate assessments were available at their schools. 

Most of the 131 teachers (n = 117, 86%) and 167 principals (n = 157, 84.4%) reported 
that a school policy for assessment accommodations existed. When asked to identify the basis 
of the policy, 66.7% (n = 74) of teachers and 79.2% (n = 118) of principals noted their 
school used state accommodations guidelines. 
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Reporting and Using Assessment Results 



The survey also included questions on school policies for the reporting and use of district 
and state assessment data (see Table 2). 



Table 2 - Reporting and Using Assessment Results 


School Policy 


Teacher 


Principal 




Response 


Response 




(Number)* 


(Number)* 


How Results Are Reported: 


(n = 1 56) 


(n = 196) 


Results not reported 


2 


2 


Individual results reported to 
Individual parents 


113 


169 


Individual results reported 
to teachers 


113 


157 


Aggregate results compiled in a 
school report to the governing board 


40 


75 


Aggregate results reported to the state 


55 


97 


Individual results reported to student's 
home district 


83 


133 


Results maintained in each student's file 


115 


162 


Don't know 


14 


6 


How Results Are Used: 

To adjust instruction or curriculum 


109 


157 


To decide on student placement within 
the school 


58 


83 


To decide on student grade- 
level promotion 


24 


41 


To decide on student's return to 
public school 


36 


50 


To evaluate teachers 


12 


20 


To identify acceptable areas of school 
performance and areas where 
improvement is needed 


101 


139 


Results not used at the school level 


23 


24 


Note: * Percentages not noted because respondents marked All that apply. 
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Teachers and principals who indicated that local district, state, or school developed assess- 
ments were used with all students (except those with exemptions) were asked how assess- 
ment results were reported and used. Approaches to reporting, in order of frequency were: 
reported to parents or guardians; reported to teachers; and maintained in each student’s file. 
Respondents also commonly noted that assessment results were used to adjust instruction and 
curriculum, to identify areas in which school performance was acceptable or needed improve- 
ment, and to make decisions about student placement within the school. About one-fourth of 
teachers and principals reported that assessment results were not used at the school level. 

Discussion 

The findings in this Issue Brief are important because they further our knowledge of 
school-level policies in day treatment and residential schools for children with EBD, and also 
provide critical information regarding accountability practices in those settings. 

Primary Accountability and Participation in Assessments 

Teachers and principals reported that the most common basis of accountability for student 
academic performance was local district and/or state assessments. However, approximately 
one-third of teachers and 20% of principals identified teacher-selected assessments as the 
primary means of accountability for student learning. In addition, about 11% of teachers and 
principals noted that their school primarily used school-developed assessments, suggesting 
that they may have little link to district and state accountability. 

Slighdy more than half of teachers and two-thirds of principals reported that most or all 
students participated in state and district the assessments, although about a third of schools 
reported having 80% or fewer students participating. However, significandy more students 
reportedly took part in district and state assessments if their school primarily served only one 
district or one state. These findings show a need for day treatment and residential schools to 
administer district and state assessments and collaborate with local and state education agen- 
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cies to assure use of appropriate assessments and full student participation. However, schools 
enrolling students from across a state or multiple states may have unique needs related to 
deciding which assessments to administer; this issue requires more investigation. 

Alternate Assessments and Assessment Accommodations 

Alternate assessments as defined under the IDEA are generally considered for a small num- 
ber of students for whom the general state or district assessments are not appropriate. Most 
typically these are students with severe cognitive disabilities who are participating in a highly 
modified or diflerentiated curriculum. For students such as those with EBD who have access 
to a general education curriculum, participation in state or district assessments is expected. 
However, students in day treatment and residential schools may have severe emotional and 
behavioral disorders and accompanying cognitive disabilities that make participation impossi- 
ble, even with accommodations. Thus, it is important to understand how these schools 
approach alternate assessments. 

In the current study, teachers and principals representing 31 states indicated that state stan- 
dardized, norm-referenced or criterion referenced assessments were available to students as 
alternate assessments. A comparison of the state alternate assessment policies of the same 31 
states (Thompson & Thurlow, 2001) revealed that only two of the states used performance 
assessments as their alternate assessments. Another six states used a combination of methods 
(e.g., body of evidence or portfolio, checklist or rating scale, analysis of lEP goals, specific 
performance assessment, combination of strategies) for obtaining alternate assessment data 
that could include a specific performance assessment. Nearly half of the states reported using 
some type of portfolio or body of evidence and another 18% noted using a checklist or rating 
scale (Thompson & Thurlow, 2001). 
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Accommodations 



Over 80% of teachers and principals reported that their school had an assessment accom- 
modation policy, most educators also noted that they used their state’s or district’s assessment 
accommodations guidelines. However, approximately 20% of day treatment and residential 
school staff reported that they offered no assessment accommodations. These findings are 
disconcerting because federal mandates require that assessment accommodations be provid- 
ed as appropriate. It is also noteworthy in light of the fact that a majority of survey respon- 
dents indicated a state department of education had accredited their school. No research cur- 
rently exists regarding accreditation requirements for day treatment and residential schools. 
However, it would be interesting to determine whether issues related to participation in 
assessments, provision of accommodations, and student performance are considered by state 
departments of education in the accreditation process of these special schools. 

Reporting and Using Assessment Results 

Teachers and principals identified three primary ways that assessment results were report- 
ed: in reports to parents or guardians; in reports to teachers; and as results maintained in 
each student’s file. A few teachers and principals did not know the school policy for reporting 
assessment results. It is encouraging that assessment results were given to parents or 
guardians and reported to districts or states. Yet, it should also be noted that a number of 
teachers and principals reported that assessment results were not used. 

The most common reported use of students’ assessment results was to adjust instruction or 
curriculum and identify areas where school performance was acceptable or needed improve- 
ment. Some teachers and principals also said that their schools used assessment data to 
make decisions regarding students’ return to their public or home schools. How they used 
the assessment data, which criteria were used, and how criteria were used to make these 
decisions is not clear. However, decisions to move students to a less restrictive environment 
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should be based primarily on the supports needed for a student to access and progress in the 
general education curriculum, not on some absolute performance level. 

Conclusions 

We know through previous research (U.S. Department of Education, 2000; Wagner, 1995) 
that academic outcomes for youth with EBD are often negative. These students typically have 
high dropout rates and difficulty maintaining employment. They may also be at risk for 
involvement with the juvenile or adult Justice systems. Day treatment and residential schools 
can provide a valuable and necessary service to these students and their families. However, to 
ensure that these schools do provide the level of supportive education students with EBD 
need, schools must be accountable for their students’ academic outcomes. 

The evidence provided in this study suggests that we have some distance to go for this to 
occur. The findings strongly suggest that more vigilance is necessary to ensure that students 
are participating in assessments with appropriate accommodations and that assessment data 
are reported and used. Local districts and state departments of education must share respon- 
sibility for including these schools in their reform efforts, holding them accountable, and pro- 
viding the necessary training and information to close this accountability gap. 
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